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SPACE VARYING GAMUT MAPPING 

1 Technical Field 

2 The technical field is color image processing using gamut correction. 

3 Background 

4 Gamut mapping is a method used to modify a representation of a color image to 

5 fit into a constrained color space of a given rendering medium. A laser-jet color printer 

6 that attempts to reproduce a color image on regular paper would have to map the 

7 photographed picture colors in a given color range, also known as the image "color 

8 gamut/' into the given printer/page color gamut. Conventional gamut mapping methods 

9 involve a pixel by pixel mapping (usually a pre-defined look-up table) and ignore the 

1 0 spatial color configuration. More recently, spatial dependent approaches were proposed 

§ i 

11 for gamut mapping. However, these solutions are either based on heurestic assumptions 

12 or involve a high computational cost. An example of such a method is discussed in 

j-y 23 "Color Gamut Mapping Based on a Perceptual Image Difference Measure," S. NakaucHi, 

y 14 et al., Color Research and Application , Vol. 24, pp. 280-291 (1999). 
^ 15 Another method relies on preserving the magnitu dgjjf gradients in an original 

D 16 image, while projecting the gradients onto the target gamut as a constraint. This multi- 

{=& 17 scale property is achieved by sampling the image around each pixel with exponentially 

?=y 1 8 increasing sampling intervals while the sampling is done along vertical and horizontal 

19 axes. The method involves modulating an L 2 measure for image difference by human 

20 contrast sensitivity functions. The method uses a model in which the contrast sensitivity 

21 function is a linear combination of three spatial band-pass filters ///, H2, H 3 , given in the 

22 spatial-frequency domain (or hi, h 2 , hs, as their corresponding spatial filters), as shown in 

23 Figure 1. 

24 For gamut mapping of the image uo in the CIELAB (which stands for Commission 

25 International de I'Eclairage (CIE) with L, A, and B (more commonly L, a, b) respectively 

26 representing converted signals for cyan (C), magenta (M) and yellow (Y)) space, the 

27 method minimizes the functional 
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E^\u a ,u b )=^ Y lfc*(p c -u c Q ]fdn, (i) 

i=\ ce{L,a,b} 

1 subject to {u L , z/, u b ) e 3 . 

2 In Equation (1), /z/ is the filter corresponding to the spectral channel c e {Z, a, 6} the i 

3 e {1, 2, 3} 'contrast sensitivity' mod, Q is the image domain, and & is the target gamut. 

4 Note that a total of nine filters are involved, three for each spectral channel and a 

5 total of three spectral channels. 

6 The filters H- are modeled by shifted Gaussians. H- is not shifted, and thus, hf 

7 is also Gaussian, while h% and h% are Gaussians modulated by two sine functions with 

8 different periods. A graphical analysis of k% and as shown in Figure 2 reveals that 

9 and approximate the derivative, operator at different scales. These two gradient 

m 

r\l 10 approximation operators are denoted by and V^ 2 . Note that any band pass filter can 

□ 

iVj 11 be considered as a version of a derivative operator. Furthermore, one possible extension 

if* 

s " 12 of the ID derivative to 2D is the gradient. Thus, the minimization of the Equation (1) 



U 1 3 functional is similar to minimizing the following functional for each channel separately. 

m 

£ J|/f * (u - u 0 f + 1 v< , (u - Wo y + 1 V^ 2 ( W - u 0 f da (2) 



14 Roughly speaking, the first term corresponds to the S-CIELAB perceptual 

1 5 measure, while the next two terms capture the need for matching the image variations at 

16 two selected scales that were determined by human perception models. One technical 

17 difficulty of the spatial filters corresponding to Equation (1) is their large numerical 

1 8 support. 

1 9 Another simple spatial-spectral measure for human color perception was proposed 

20 in "A Spatial Extension of CIELAB for Digital Color Image Reproduction," Zhang et al., 

21 Proceedings of the SID Symposium , Vol. 27, pp. 731-34, (1996). The c S-CIELAB 5 

22 defines a spatial-spectral measure for human color perception by a composition of spatial 

23 band-pass linear filters in the opponent color space followed by the CIELAB Euclidean 

24 perceptual color. 
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Summary 



Gamut mapping is a method to modify representation of a color image to fit into a 
constrained color space of a given rendering medium. A laser-jet color printer that 
attempts to reproduce a color image on regular paper would have to map the 
photographed picture colors in a given color range, also known as the image "color 
gamut, " into the given printer/page color gamut. 

A method an apparatus uses a variational approach for space dependent gamut 
mapping. The method presents a new measure for the problem. A solution to the 
formulation according to the method is unique if the gamut of the target device is known. 
A quadratic programming efficient numerical solution provides real-time results. 

In an embodiment, the method comprises receiving an input image, converting 
color representations of an image pixel set to produce a corresponding electrical values 
set, applying a space varying algorithm to the electrical values set to produce a color- 
mapped value set, and reconverting the color-mapped value set to an output image. The 
space varying algorithm minimizes a variational problem represented by 



E(u) = J n [D 2 + a\ VD\ 2 }lQx , subject to u e 3 , where Q is a support of the image, 

a is a non-negative real number, D=g*(u-Uo), and g is a normalized Gaussian kernel with 
zero mean and a small variance a. 

The method may be implemented in a variety of image capture reproduction 
devices, including cameras and printers. The method may be embodied on a computer- 
readable medium, such as a CD-ROM, for example. 
Description of the Drawings 

The detailed description will refer to the following drawings, in which like 
numerals refer to like objects, and in which: 

Figure 1 is a qualitative description of filters modeling the human contrast 
sensitivity functions in the spectral domain; 

Figure 2 illustrates a shifted Gaussian; 

Figure 3 is a block diagram of an apparatus that uses a variational approach for 
gamut mapping; 

Figures 4-7 illustrate algorithms and processes used with the apparatus of Figure 




3; 



Figure 8 illustrates an example of a behavior of an algorithm used by the 
apparatus of Figure 3; 
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1 


Figure 9 is a flowchart of an algorithm illustrating an alternative operation of the 




2 


apparatus of Figure 3; and 




3 


Figure 10 is a block diagram of a computer system that implements the algorithm 




4 


of Figure 4. 




5 


Detailed Description 




6 


Gamut mapping is a method used to modify a representation of a color image to 




7 


fit into a constrained color space of a given rendering medium. A laser-jet color printer 




8 


that attempts to reproduce a color image on regular paper would have to map the 




9 


photographed picture colors in a given color range, also known as the image "color 




10 


gamut," into the given printer/page color gamut. Conventional gamut mapping methods 




11 


involve a pixel by pixel mapping (usually a pre-defined look-up table) and ignore the 


o 


12 


spatial color configuration. More recently, spatial dependent approaches were proposed 


13 


for gamut mapping. However, these solutions are either based on heurestic assumptions 


m 

£3-=, 


14 


* 1 1*1 J. A* 1 J_ 

or involve a high computational cost. 


5 y 


15 


The gamut mapping problem is related to the Retinex problem of illumination 


M 

« . 2 


16 


compensation and dynamic range compression. The basic Retinex problem is: How to 


i™ 

i; 


17 


estimate the reflectance image from the given acquired image c A reasonable optical 




18 


model of the acquired image S asserts that it is a multiplication of the reflectance R and 


u = 

{*& 


19 


the illumination L images. Where the reflectance image is a hypothetic image that would 


o 


20 


have been measured if every visible surface had been illuminated by a unit valued white 




21 


illumination source, and the illumination image is the actual illumination shaded on 




22 


surfaces in the scene. In the log domain: 



23 s = r + / 

24 where s, r, and / are the respective logarithms of S, R, and Z. Since surface patches can 

25 not reflect more light than has been shaded on them, R < 1 =o r < 0. Thus, in an ideal 

26 situation, image r < 0, which is perceptually similar to s. For the Retinex, an additional 

27 physically motivated constraint is provided, namely, that the illumination image / = s-r 

28 is smooth, i.e. the gradient |v/| = |V(r-s)| is small. But this is just another way to say 

29 that the features of r are similar to those of s, since the illumination is assumed not to 

30 create perceptual features in s. In the gamut mapping problem an image uo exists, and the 

3 1 problem is to estimate an image u e 3 , which is not only perceptually similar to uo, but 

32 also has similar perceptual features as uq. 
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A good measure of image deviation captures the perceptual difference between 
the initial, uq and the final, u, images. This is modeled by 

D = g*{u-u 0 } (3) 

where g may be a normalized Gaussian kernel with zero mean and a small variance a . 
This model is good for small deviations. However, for large deviations it should be 
elaborated to account for possible perceptual feature differences, which may be modeled 
by the difference of gradients, which due to linearity, turns out to be the gradient of 
Equation (3) 

VD = V[g * (u - u Q )] = g * (Vw - Vw 0 ) (4) 

The proposed measure yields the functional 

E(u) = \(d 2 + a\VD\ 2 )/Q, (5) 
n 

subject to we i9 5 
which should be minimized. 

Taking a first variation of Equation (5) with respect to u yields the Euler-Lagrange 
equation: 

= g * (adiv(y g * (u - w 0 ))- g * (u - w 0 ))= 0, (6) 

ou 

Reformulating Equation (6) as a gradient descent flow for u, provides the following 
minimization scheme: 

du . _ _ 

— = ag*AD-g*D ^ 

The functional (Equation (5)) and the resulting minimization scheme are both 
Euclidean invariant in the image plane. They are thus both translation and rotation 
invariant. As the parameter a goes to zero, the S-CIELAB model is approximated, while 
for effective a the result is a proper extension to the perceptual measures, with an 
efficient numerical implementation. 
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To provide a numerical solution, recall that an artificial time parameter t was 
added to the image w(x, y\ which now reads u(x, y 9 /). A first step is to discretize the 
Eueler-Lagrange gradient descent equation, by first taking a simple forward explicit 
approximation for the t derivative, 

u n+l -u n 

= ag* AD - g* D 

T 

where r = dtandu"(x J y)& u(x 9 y;nr). 

Next, the space derivatives are solved. Let w," ~ u(ih,jh 9 nf) 9 where uniform 

spatial spacings are assumed in the x and y directions of size h. Using central derivatives 
in space, 

h 2 



and the same for the y direction. Using the relation g * (g * u) = g x * g x * w, the 
algorithm next computes the kernels D 2 = g x * g x + g y * g y = Dxg * Dxg + D y g * D y g. The 
explicit approximation reads: 

D" = g*g*{u"-u Q ) 
L" = Z) 2 *^- Wo ) 

subject to the constraint e »9. 

To speed up convergence, a standard coarse to fine pyramidal approach may be 
preferred. For example, an original image S may be composed of a set of 5 12 x 5 12 
pixels. The image S may then be averaged down (sub-sampled or decimated) by taking, 
for example, every 4 adjacent pixels and replacing the block of four pixels with a single 
pixel having a value that represents an average of the four pixels. This results in an image 
set that is 256 x 256 pixels. The sub-sampling can be continued, yielding a subsequent 
reduced pixel set, or layer, comprising 128 x 128 pixels. This process may continue, with 
each resulting layer being an increasingly coarser version of the original image S, The 
final layer will be a set of only 2x2 pixels, and likely will be blurred. The process of 



u x * D x u = 
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sub-sampling or decimating to provide increasingly coarser details results in an image 
pyramid. The original image S represent the base of the image pyramid at 512x512 
pixels, followed by layers of 256 x 256 pixels, 128 x 128 pixels, 64 x 64 pixels, and so 
on. Using this approach of an image pyramid allows for rapid convergence of the 
solution to the functional of Equation (5). The process starts by using the coarsest layer 
of the image pyramid, and running a number of iterations until a solution is achieved. 
The solution is then used to initialize the next finer layer of the image pyramid, resulting 
in less computation in order to arrive at a solution. The process continues until the finest 
layer (e.g., the original, full resolution, image set for S) is reached. As an alternative to 
the image pyramid, a full multi-grid method may be used. 

The functional of Equation (5) has a Quadratic Programming (QP) form, since the 
penalty term is quadratic and the constraint is linear. If the set 9 is convex, the overall 
problem is convex if and only if the Hessian of the functional Equation (5) is positive 
definite. In such a case, there is a unique local minimum that is also the global solution to 
the problem. In the above embodiment, the Hessian is given by g * (1 - a A) * g, which is 
positive definite for all a > 0. Thus, for a convex target gamut 9 , there exists a unique 
solution. 

The above-described functional (Equation (5)) may be used in imaging devices to 
map a gamut of an image to the gamut of a device that attempts to reproduce the image. 
Figure 3 is a block diagram of an apparatus 100 that may be used for gamut mapping 
according to the functional of Equation (5). An image capture device 101 receives an 
input original image and produces an output image S 1 02, which is an electrical signal 
representing a colorimetric value image. For example, the capture device 101 may 
convert at least three colorimetric values of each pixel of the original image into 
corresponding electrical signals. The electrical signals may indicate the L, a, b values, for 
example. Other colorimetric values may be the XYZ tristimilus value and the L, U, V or 
device dependent RGB values. A gamut mapper 105 produces an in-gamut image S' 106. 
Finally, a rendering module 107 provides a rendered image S" 108. The rendering 
module 107 may be implemented as a color laser printer. The thus-generated image S" 
108 may represent a best-fit image, given gamut limitations of the device in which the 
image reproduction is to occur. 

The image S 102 may represent the image as sensed by the capture module 103. 
The gamut mapper 105 applies an algorithm to extract and map the values of the image S 
102 into the gamut of the image reproduction device 107. In particular, the gamut 
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mapper 105 may apply an algorithm that solves the problem represented by Equation (5), 
thereby solving the gamut mapping problem and optimizing the output image S" 108. 

Figure 4 is a block diagram showing routines of an algorithm 120 that may be 
used to complete the gamut mapping function. In an embodiment, the algorithm 120 
provides a numerical solution to the functional of Equation (5). 

The algorithm 120 begins with input block 121, where an input to the algorithm 
120 is an image S of size [N, M], and the two parameters a and p. 

In initialization block 123, a Gaussian pyramid of the image s is computed. The 
thus-constructed pyramid contains p resolution layers (from fine (1) to coarse (p)) with 
the current resolution layer (k) set to p. In block 125, Tk iterations of a gradient descent 
algorithm are applied to the resolution layer, until all resolution layers are checked, 
block 127. In block 129, the next resolution layer is updated. When all resolution layers 
are processed, the result is the final output of the algorithm 120. 

Figure 5 is a block diagram showing the initialization routine 123 in more detail. 
In block 131, a counter is initialized with i = 1. In block 132, the Gaussian pyramid is 
constructed by smoothing the image with a kernel, such as the kernel kpm\ 
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In block 133, the image is decimated by, for example, a 2:1 ratio. The process is repeated 
(block 134) and the counter is incremented (block 135) p times where p < lg2 (min (M, 

N)). This process produces a sequence of images {S k } p k=x conventionally named the 

"Gaussian Pyramid of S" The image Si is the original image S, and S p is an image with 
the coarsest resolution for the Gaussian pyramid. 

In block 137, the process sets k = p 9 i.e., starts at the coarsest resolution layer k, 
and sets the initial condition Lo = max {S p }. 

Figure 6 is a block diagram of an embodiment of the processing main routine 125. 
The routine 125 starts at block 141, where the new resolution layer is initialized and Tk 
and otk are set (e.g., Tk = K*Tq). 
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Then, for/ = 1, .., T k: 

In block 143, the routine 125 calculates the gradient: 

1 G = A(w -u 0 )+a k (u-u 0 ) 

2 where Ax is the convolution of each of the color planes of x with Klap: 



K LAP - 



0 1 0 

1 -2 1 
0 1 0 



and a k is, for example a k = a *2 2(k-1) 



In block 145, the routine 125 calculates jj msd' 



fa 

IB 
IV 
p 



Mnsd ~ ZC /^;(g * AG>+ a K TG 2 ) 



6 In block 147, the routine 125 completes the gradient descent iteration: 

L. = Lj_ { — /u Q - fi NSD - G, 
1 Where y G is a constant; for example y o = 0.8. 

8 In block 148, the result is projected onto the constraints: 

Lj =Proj^(z,) 

9 Where Proja(x) is a projection of x into the gamut Q. 

10 In block 149, if j ^ Tk, processing returns to block 143. Otherwise, the routine 

11 125 ends. 

12 Returning to Figure 4, in block 127, if k > 1, the result L T k is up scaled (2:1 ratio) 

13 by pixel replication into the new Lo, the initialization for the next resolution k - 1 layer. 

14 The resolution layer is updated k = k- 1 , and the algorithm proceeds by going again to 

1 5 block 125. If k = 1, the result Ln, is the final output of the algorithm 120. 

16 Figure 7 is a block diagram of an alternative embodiment of an algorithm routine, 

17 denoted as 120a. The routine 120a may be applied to solve the equation: 

18 »r =«;+r(«i.;-^) 

19 subject to the constraint u"j e 9, 
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1 The routine 120a begins with block 151, where the kernels D 2 and g, and the 

2 counter k are initialized. 

3 In block 153, the routine 120a calculates: 

4 D" =g*g*(u" -u Q ). 

5 In block 155, the routine 120a calculates: 

6 L n =D 2 *(w" -u 0 ). 

7 In block 157, the routine 120a performs the gradient steps to solve u"/ 1 as noted 

8 above. 

9 In block 158, the routine 120a determines if k>l, and if so, processing moves to 

10 block 1 59, where k is set to k-1 . Processing then returns to block 1 53. Otherwise, the 

11 routine 120a ends. 

Iq 12 The penalty function as shown in Equation (5) tends to create halos in the 

ji 13 resulting image. Figure 8 explains the origin of those halos through a one dimensional 

fy 14 example. Figure 8 shows an original signal 160 that is outside of gamut 162 (delimited in 

O 

y 15 the gray range by dotted lines 163 and 164). Projecting the signal 160 onto the gamut 162 

16 will result in a constant value, denoted as "High" (163) and loss of all detail. Figure 8 

v 

P 17 also shows the result of processing the original signal 160. Dashed line 165 represents 

3er - 

18 the result of scaling the signal 160 into the allowed range of the gamut 162. All the 

jif 19 details are preserved, but with a smaller contrast. As opposed to point operations, the 

.M 20 space dependent approach represented by Equation (5) yields a signal 166 (the solid line) 

21 that preserves the details with high contrast. However, halos occur near strong edges 167, 

22 which means that near the edges 1 67 there is a slow transition from low to high values. 

23 In order to avoid these phenomena, the penalty term may be modified, and robust 

24 estimation tools may be used. The original penalty term in Equation (5) may be replaced 

25 by: 

E{u)= j(p,{D)+ap 2 (\VD\))dCl. (8) 

26 which for p l (x)= p 2 ( x ) = x2 coincides with Equation (5). If the function p(x) grows 

27 slower than x 2 as x— > oo, behavior near strong edges improves. Good candidates for p(x) 



28 are p{x) ~ \x\ or p(^)~ vl + x 2 
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A different and simpler (linear) approach with similar robust behavior involves 
solving the original Equation (5) twice, with two different values of a. A solution with a 
small a may be denoted as u sma ii and a solution that corresponds to the high value of a 
may be denoted as w/,^. The solution u sma u has smaller contrast at areas with small 
details, yet has almost no halos. On the other hand, the solution Uhi g h preserves the small 
details, but at the expense of strong halo effects. By averaging these two results (u sma u 
and Uhigh) in a spatially adaptive way provides a simple, improved solution. The 
improved solution is therefore: 

U final small 

The weight w[k, j] should be close to one near strong edges, and close to zero in relatively 
smooth regions. In an embodiment, 

w\k, /1= 5- 

L,n \ + /?\Vg*u 0 \ 2 

provides a robust estimation. 

Halo problems have been recently dealt with in relation to Dynamic Range 
Compression. Solutions proposed included anisotropic diffusion and robust filtering. 
The halo-related solutions described herein are solutions to the same halo problem. 

Figure 9 is a flowchart showing an operation 180 of the gamut mapper 105. The 
process begins in start block 185. In converter block 195, the raw image S is converted 
from electrical signals having arbitrary values (colorimetric values) to locally defined 
color descriptors. 

In block 200, the converted image signal is decimated to form an image pyramid. 
A 2: 1 decimation scheme may be used, for example. Subsequent steps in the operation 
1 80 begin with the coarsest resolution layer of the image pyramid using a space varying 
algorithm. In block 205, the gamut mapper 105, using a space varying algorithm such as 
that represented by Equation (5) calculates the image deviation for the coarsest resolution 
layer. This process may involve calculating a first variation (block 2 1 0), determining a 
gradient descent flow (block 215), and solving the resulting gradient subject to a 
constraint (block 220). In block 225, a determination is made if the current resolution 
layer is the last (finest) resolution layer. If additional layers remain for processing, the 
process 180 returns to block 205. Otherwise, the process 180 moves to block 230, where 
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local color descriptors are converted to the required output format. In block 245 the 
process 180 ends. 

The above-described space-varying algorithm 120, as represented by Equation (5), 
for example may be executed using a general purpose computer system 250 as shown in 
Figure 10. The computer system 250 includes an input device 260, a computer unit 270, 
a display 280, a keyboard 290, and a storage device 300. The storage device 300 may be 
a CD-ROM drive, for example. Program code instructions for executing the algorithms 
120, 120a and 180 may be stored on a CD-ROM 305, or other computer readable 
memory, which is read by the storage device 300. The program code read out of the CD- 
ROM 305 are executed by a CPU of the computer unit 270. The CPU may perform the 
processing shown by the flowchart of Figure 9. The algorithms 120, 120a, and 180 also 
may be performed in a camera or printer hardware. 
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